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Preface 


In  2001,  research  was  undertaken  to  assess  the  amount  and  type  of  derogatory  in¬ 
formation  yielded  by  sources  used  in  Single-Scope  Background  Investigations — Periodic 
Reinvestigations  (SSBI-PRs).  Analyses  showed  that  some  sources  used  in  periodic  rein¬ 
vestigations  are  far  more  productive  than  others  and  that  the  most  productive  sources  can 
be  used  to  identify  cases  in  which  less  productive  sources  are  most  likely  to  yield 
derogatory  information. 

Building  upon  these  research  findings,  PERSEREC  conducted  a  follow-up  study 
to  assess  the  feasibility  of  a  two-phase  periodic  reinvestigation.  In  a  two-phase  SSBI-PR, 
Phase  2  sources  are  used  only  in  cases  in  which  Phase  1  sources  yield  derogatory  infor¬ 
mation,  thereby  suggesting  that  further  investigation  is  warranted.  Phasing  was  shown  to 
save  substantial  resources  with  minimal  loss  of  derogatory  information.  Additional 
phasing  research  conducted  by  PERSEREC  and  other  organizations  supported  the  viabil¬ 
ity  of  phasing. 

This  report  presents  the  results  of  an  evaluation  of  a  1-year  implementation  of  a 
phased  SSBI-PR  by  the  Defense  Security  Service  (DSS).  Findings  of  this  study  indicate 
that  phasing  results  in  a  significant  reduction  in  resource  requirements  with  very  little 
loss  of  derogatory  information.  Taken  together,  findings  of  phasing  research  conducted 
by  PERSEREC  suggest  that  the  investigative  standards  for  conducting  background  in¬ 
vestigations  should  be  changed  such  that  a  two-phase  SSBI-PR  becomes  the  new  mini¬ 
mum  requirement  for  conducting  periodic  reinvestigations. 


James  A.  Riedel 
Director 
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Executive  Summary 


Research  conducted  by  the  Defense  Personnel  Security  Research  Center 
(PERSEREC)  has  shown  that  a  two-phase  Single-Scope  Background  Investigation-Peri¬ 
odic  Reinvestigation  (SSBI-PR)  results  in  a  more  effective  use  of  investigative  resources. 
In  a  two-phase  periodic  reinvestigation,  not  all  sources  are  exploited  in  all  cases.  Phase  2 
sources  (listed  and  developed  references,  residence  interviews,  and  residence  records)  are 
used  only  when  “issue-relevant  information”  is  yielded  by  one  or  more  Phase  1  sources 
(all  other  sources  in  the  SSBI-PR).  Issue-relevant  information  is  information  that  is  rele¬ 
vant  to  establishing  an  issue  of  potential  current  security  concern. 

In  the  summer  of  2002,  the  Deputy  Assistant  Secretary  of  Defense  (Security  and 
Information  Operations),  in  conjunction  with  the  Intelligence  Community,  approved  a  1- 
year  test  implementation  of  the  phased  SSBI-PR.  In  FY03,  the  Defense  Security  Service 
(DSS)  implemented  the  phased  periodic  reinvestigation.  To  assess  the  performance  of  the 
phased  SSBI-PR  as  implemented  by  DSS,  the  current  evaluation  study  was  undertaken. 

Data  Used  in  This  Study 

In  FY03  DSS  conducted  15,668  SSBI-PRs  under  the  phasing  model.  From  a  sub¬ 
set  of  these  cases  (SSBI-PRs  that  closed  between  October  1,  2002  and  March  31,  2003), 
DSS  selected  a  random  sample  of  391  Phase  1  SSBI-PRs.  DSS  reopened  these  Phase  1 
cases  and  conducted  Phase  2  sources  for  them  so  PERSEREC  could  determine  whether 
Phase  2  leads  would  have  yielded  issue-relevant  information  if  they  had  been  used  ini¬ 
tially.  (Because  these  cases  were  designated  as  Phase  1  by  DSS,  Phase  2  sources  would 
not  normally  have  been  included.)  DSS  also  provided  PERSEREC  with  a  convenience 
sample  of  214  Phase  2  SSBI-PRs — cases  in  which  both  Phase  1  and  Phase  2  sources  were 
included  in  the  investigation  prior  to  its  close. 

Evaluation  Criteria 

We  evaluated  the  performance  of  phasing  using  five  outcome  measures.  We  com¬ 
pared  the  proportion  of  SSBI-PRs  that  DSS  expanded  with  the  rate  of  case  expansion 
found  in  previous  studies.  We  determined  the  percentage  of  Phase  1  SSBI-PRs  (cases 
DSS  designated  as  Phase  1)  in  which  Phase  2  sources  would  have  yielded  issue-relevant 
information  if  they  had  been  used.  (Because  these  sources  would  not  normally  have  been 
included,  the  information  they  yielded  would  have  been  missed.)  We  calculated  our 
evaluators’  level  of  agreement  with  DSS’  Phase  1  designation  and  we  calculated  our 
evaluators’  level  of  agreement  with  DSS’  Phase  2  designation.  (Our  evaluators  had  ex¬ 
tensive  experience  conducting  background  and  counterintelligence  investigations  and 
were  trained  in  the  application  of  phasing  expansion  criteria.)  Finally,  we  estimated  the 
reduction  in  resource  requirements  that  resulted  from  DSS’  implementation  of  phasing. 


Key  Findings 

Using  the  entire  population  of  SSBI-PRs  conducted  under  the  phasing  model  in 
FY03  (N=15,668),  we  determined  the  proportion  of  SSBI-PRs  that  DSS  expanded  to  in¬ 
clude  Phase  2  sources,  and  compared  this  expansion  rate  with  that  found  in  prior 
research. 

•  Using  the  expansion  criterion  of  issue-relevant  information  in  one  or  more  Phase 
1  sources,  2,341  of  the  15,668  SSBI-PRs  conducted  by  DSS  under  the  phasing 
model  were  designated  as  Phase  2  investigations  constituting  an  expansion  rate  of 
15%.  Conversely,  85%  of  the  SSBI-PRs  subject  to  phasing  were  deemed  by  DSS 
investigators  to  contain  no  issue-relevant  information  in  Phase  1  sources  and  were 
not  expanded  to  include  Phase  2  sources. 

•  Our  evaluators  would  have  expanded  16%  of  SSBI-PRs  in  comparison  with  DSS’ 
expansion  rate  of  15%. 

•  Both  the  DSS  expansion  rate  of  15%  and  our  evaluators’  expansion  rate  of  16% 
are  lower  than  that  shown  in  prior  research  in  which  23%  of  DoD  SSBI-PRs  were 
deemed  to  contain  issue-relevant  information  in  one  or  more  Phase  1  sources. 

For  the  sample  of  Phase  1  investigations  included  in  our  evaluation  (n=391),  DSS 
conducted  Phase  2  leads  after  the  investigations  closed  so  PERSEREC  could  determine 
the  information  that  was  missed  by  not  using  these  sources. 

•  In  387  of  these  391  SSBI-PRs,  Phase  2  sources  yielded  no  issue-relevant 
information. 

•  In  four  of  these  391  SSBI-PRs,  Phase  2  sources  would  have  yielded  issue-relevant 
information. 

•  The  four  “missed”  SSBI-PRs  were  examined  and  the  issue-relevant  information 
that  was  yielded  by  Phase  2  sources  was  found  to  be  relatively  minor.  The  infor¬ 
mation  yielded  by  Phase  2  sources  in  the  missed  SSBI-PRs  is  described  in  the 
body  of  this  report. 

Our  evaluators  reviewed  Phase  1  sources  in  the  sample  of  Phase  1  investigations 
included  in  our  evaluation  (n=391)  to  determine  whether  they  felt  these  sources  yielded 
is  sue -relev ant  information  and  hence,  whether  any  of  these  cases  should  have  been 
expanded. 

•  Of  the  391  SSBI-PRs  that  DSS  designated  as  Phase  1,  our  evaluators  agreed  that 
370  should  not  have  been  expanded  but  believed  that  21  should  have  been  ex¬ 
panded  to  include  Phase  2  sources.  Thus,  our  evaluators  agreed  with  DSS’  Phase 
1  designation  95%  of  the  time. 
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•  In  expanding  an  additional  21  SSBI-PRs,  our  evaluators  would  have  expanded 
three  of  the  four  SSBI-PRs  that  DSS  did  not  expand  in  which  Phase  2  sources 
would  have  been  productive. 

Our  evaluators  reviewed  Phase  1  sources  in  the  sample  of  Phase  2  investigations 
(n=214)  to  determine  whether  these  sources  yielded  issue-relevant  information  and, 
hence,  whether  these  cases  should  have  been  expanded. 

•  Of  the  214  SSBI-PRs  that  DSS  designated  as  Phase  2,  our  evaluators  agreed  that 
172  should  have  been  expanded  but  believed  that  42  should  not  have  been  ex¬ 
panded  to  include  Phase  2  sources.  Our  evaluators  agreed  with  DSS’  Phase  2 
designation  80%  of  the  time. 

•  In  deciding  against  the  expansion  of  42  SSBI-PRs  that  DSS  expanded,  our  evalua¬ 
tors  would  have  increased  the  number  of  missed  cases  by  one  SSBI-PR.  If  our 
evaluators  had  made  the  phasing  decision,  they  would  not  have  expanded  one 
SSBI-PR  that  DSS  expanded  in  which  Phase  2  sources  yielded  issue-relevant 
information. 

Using  data  gathered  for  a  cost  study  conducted  by  DSS,  data  gathered  for  the  ini¬ 
tial  phasing  study,  and  recent  analyses  conducted  to  measure  SSBI-PR  time  requirements, 
we  estimated  the  reduction  in  resource  requirements  that  resulted  from  DSS’  implemen¬ 
tation  of  a  two-phase  SSBI-PR  in  FY03.  Our  analyses  showed  that  whatever  the  cost  of 
conducting  the  15,668  periodic  reinvestigations  in  FY03  would  have  been  without  the 
implementation  of  phasing,  in  implementing  a  phased  SSBI-PR,  DSS  reduced  these 
resource  requirements  by  approximately  22%. 

Findings  of  this  evaluation  study  support  findings  of  prior  phasing  research  in 
which  phasing  is  shown  to  result  in  substantial  resource  savings  with  minimal  loss  of  de¬ 
rogatory  information.  In  the  small  number  of  cases  that  were  not  expanded  by  DSS  or  by 
our  evaluators  in  which  Phase  2  sources  would  have  been  productive  (four  SSBI-PRs  and 
two  SSBI-PRs,  respectively),  the  issue -relevant  information  that  was  missed  was  rela¬ 
tively  minor.  Findings  of  this  study  suggest  that  phasing  operates  effectively  even  when 
there  is  some  level  of  disagreement  as  to  which  SSBI-PRs  should  be  expanded.  Phasing 
operated  effectively  when  DSS  investigators  made  the  phasing  decision  (as  in  this  study 
and  in  the  DoD  pilot  test  conducted  in  2001)  and  also  when  different  sets  of  evaluators 
made  the  phasing  decision. 
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Introduction 


In  the  summer  of  2002,  the  Deputy  Assistant  Secretary  of  Defense  (Security  and 
Information  Operations),  in  conjunction  with  the  Intelligence  Community,  approved  a 
1-year  test  of  a  two-phase  Single-Scope  Background  Investigation — Periodic  Reinvesti¬ 
gation  (SSBI-PR).  The  phased  SSBI-PR  was  subsequently  implemented  by  DSS  for  all 
periodic  reinvestigations  conducted  in  FY03.  The  current  study  evaluates  the  perform¬ 
ance  of  phasing  as  implemented  by  DSS. 

Background 

In  2001,  a  six-person  team  of  experienced  personnel  security  adjudicators  and  in¬ 
vestigators  reviewed  4,721  case  files  at  four  agencies  routinely  conducting  or  contracting 
out  the  Single-Scope  Background  Investigation — Periodic  Reinvestigation  (SSBI-PR): 
Department  of  Defense  (DoD),  Central  Intelligence  Agency  (CIA),  National  Reconnais¬ 
sance  Office  (NRO),  and  Office  of  Personnel  Management  (OPM).  Information  was 
gathered  pertaining  to  Subjects’  backgrounds  and  the  seriousness  and  amount  of  infor¬ 
mation  yielded  by  sources  in  these  SSBI-PRs. 

Analyses  showed  that  some  sources  used  in  periodic  reinvestigations  are  far  more 
productive  than  others.  Sources  that  yielded  the  greatest  amounts  of  derogatory  informa¬ 
tion  were  the  same  for  the  four  organizations.  The  three  most  productive  sources  were  the 
personnel  security  questionnaire,  the  subject  interview,  and  the  credit  report.  These  three 
sources,  when  combined,  yielded  “issue-relevant  information”  in  nearly  all  periodic  rein¬ 
vestigations  in  which  any  other  sources  yielded  issue-relevant  information.  Issue-relevant 
information  is  information  that  is  relevant  to  establishing  an  issue  of  potential  current  se¬ 
curity  concern.  These  three  sources,  together,  identified  87%  to  95%  of  SSBI-PRs  with 
one  or  more  items  of  issue-relevant  information  and  identified  92%  to  100%  of  cases 
upon  which  the  organizations  took  administrative  action. 

The  findings  of  this  study,  documented  in  SSBI-PR  Source  Yield:  An  Examination 
of  Sources  Contacted  During  the  SSBI-PR  (Kramer,  Crawford,  Heuer  &  Hagen,  2001), 
provided  a  starting  point  for  designing  an  improved  SSBI-PR  process.  Building  upon  the 
findings  of  the  productivity  of  sources  research,  a  second  study  was  conducted  to  test  the 
feasibility  of  a  two-phase  periodic  reinvestigation. 

The  Two-Phase  SSBI-PR 

Using  data  gathered  for  the  initial  study,  analyses  were  conducted  to  explore  the 
feasibility  of  a  two-phase  SSBI-PR.  In  a  phased  reinvestigation  not  all  sources  are  ex¬ 
ploited  in  all  cases.  Phase  2  sources  are  used  only  when  issue-relevant  information  is 
yielded  by  one  or  more  Phase  1  sources,  thereby  suggesting  that  further  investigation  is 
warranted.  The  phasing  approach  is  similar  to  medical  screening  where  findings  of  initial 
tests  determine  if  follow-up  tests  are  appropriate.  The  SSBI-PR  phasing  model  is  illus¬ 
trated  in  Figure  1. 
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Figure  1  The  Two-Phase  SSBI-PR. 

If  Phase  1  sources  do  not  yield  issue-relevant  information,  the  investigation  is 
concluded.  Thus,  Phase  1  SSBI-PRs  contain  Phase  1  sources  only.  If  Phase  1  sources 
yield  issue-relevant  information,  the  investigation  is  expanded  to  include  Phase  2  sources. 
Thus,  Phase  2  SSBI-PRs  include  both  Phase  1  and  Phase  2  sources  (Figure  2). 


Figure  2  Sources  Used  in  Phase  1  and  Phase  2  SSBI-PRs. 

In  the  initial  phasing  study,  several  different  models  were  tested.  In  the  optimal 
phasing  model,  Phase  1  sources  included  all  sources  currently  used  in  the  SSBI-PR  ex¬ 
cept  listed  and  developed  references,  residence  interviews,  and  residence  records.  These 
sources  are  less  productive  in  general  and  are  especially  unproductive  in  periodic  rein¬ 
vestigations  in  which  no  Phase  1  sources  yield  issue-relevant  information  (Kramer,  Craw¬ 
ford,  Heuer  &  Hagen,  2001). 

Findings  of  the  phasing  analyses,  documented  in  A  New  Approach  to  the  SSBI- 
PR:  Assessment  of  a  Phased  Reinvestigation  (Heuer,  Crawford,  Kramer  &  Hagen,  2001), 
show  that  phasing  saves  substantial  resources  with  minimal  loss  of  derogatory  informa¬ 
tion.  Findings  of  the  initial  phasing  study,  a  subsequent  DoD  pilot  test  of  phasing  (see 
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Heuer,  Crawford,  Kirkpatrick  &  Kramer,  2002),  and  independent  analyses  conducted  by 
CIA  and  NRO  showed  phasing  to  be  feasible  for  DoD,  OPM,  CIA,  and  NRO  SSBI-PRs 
alike. 


In  the  summer  of  2002,  the  Deputy  Assistant  Secretary  of  Defense  (Security  and 
Information  Operations),  in  conjunction  with  the  Intelligence  Community,  approved  a  1- 
year  test  of  the  phased  SSBI-PR.  The  phased  SSBI-PR  was  subsequently  implemented  by 
DSS  for  all  periodic  reinvestigations  conducted  in  FY03. 

Purpose  of  the  Evaluation 

In  this  evaluation,  DSS’  implementation  of  a  phased  SSBI-PR  is  assessed  in  five 
ways.  We  determined  DSS’  rate  of  case  expansion  (the  percent  of  SSBI-PRs  DSS  desig¬ 
nated  as  Phase  2)  and  compared  this  with  the  expansion  rate  found  by  our  subject  matter 
experts.  We  estimated  the  percentage  of  Phase  1  SSBI-PRs  in  which  Phase  2  sources 
would  have  been  productive  had  they  been  used  in  the  original  investigation.  We  docu¬ 
mented  the  extent  to  which  two  experts  with  extensive  investigative  experience  agreed 
with  DSS’  Phase  1  and  Phase  2  designations,  and  examined  the  repercussions  of  the 
phasing  decisions  made  by  these  evaluators.  We  estimated  the  resource  savings  that 
occurred  as  a  result  of  DSS’  implementation  of  phasing.  Our  primary  research  questions 
are  shown  in  Table  1. 


Table  1 

Research  Questions  Answered  to  Evaluate  DSS’ 
Implementation  of  a  Phased  SSBI-PR 


1.  What  percentage  of  SSBI-PRs  was  expanded  by  DSS  compared  with  our  evaluators’ 
case  expansion  rate? 


2.  In  what  percentage  of  Phase  1  SSBI-PRs  (cases  that  DSS  did  not  expand  to  include 
Phase  2  sources)  would  Phase  2  sources  have  yielded  issue-relevant  information  if 
they  had  been  used?1' 


3.  Of  the  SSBI-PRs  that  DSS  did  not  expand  to  include  Phase  2  sources,  what 
percentage  did  our  evaluators  agree  should  not  have  been  expanded? 


4.  Of  the  Phase  2  SSBI-PRs  (cases  that  DSS  expanded  to  include  Phase  2  sources), 
what  percentage  did  our  case  evaluators  agree  should  have  been  expanded? 


5.  What  reduction  in  resource  requirements  resulted  from  DSS’  implementation  of  a 
phased  SSBI-PR? 


aDSS  conducted  Phase  2  leads  for  our  sample  of  Phase  1  investigations  after  the  cases  were  closed  so  it 
would  be  possible  to  determine  the  number  of  cases  in  which  Phase  2  sources  would  have  yielded  issue¬ 
relevant  information  if  they  had  been  used. 
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Methodology 

Sample  Characteristics 

In  FY03,  DSS  conducted  a  total  of  15,668  SSBI-PRs  under  the  phasing  model. 
The  rate  at  which  DSS  expanded  SSBI-PRs  was  calculated  using  all  FY03  cases,  but  only 
a  sample  of  these  cases  were  reviewed  and  coded  by  our  evaluators  to  answer  our  other 
research  questions. 

From  the  subset  of  SSBI-PRs  that  DSS  closed  between  October  1,  2002  and 
March  31,  2003  (n=2,233),  DSS  selected  a  random  sample  of  391  Phase  1  cases.  DSS 
then  conducted  Phase  2  sources  on  these  investigations  so  PERSEREC  could  determine 
whether  Phase  2  sources  would  have  yielded  issue-relevant  information  if  they  had  been 
used.  Because  these  cases  were  designated  as  Phase  1  cases  by  DSS,  Phase  2  sources 
would  not  normally  have  been  included.  DSS  also  provided  PERSEREC  with  a  conven¬ 
ience  sample  of  214  Phase  2  SSBI-PRs  (cases  in  which  both  Phase  1  and  Phase  2  sources 
were  included  in  the  investigation).  Characteristics  of  the  Phase  1  and  Phase  2  SSBI-PR 
samples  are  shown  in  Table  2. 


Table  2 

Sample  Characteristics 
(n=605) 


Phase  1  SSBI-PRs  Phase  2  SSBI-PRs 
( n=391 )  (n=214) 


n 

% 

n 

% 

Age 

17-30 

64 

16 

47 

22 

31-40 

159 

41 

79 

37 

41-50 

98 

25 

58 

27 

51-60 

61 

16 

22 

10 

61  or  older 

9 

2 

8 

4 

Sex 

Female 

51 

13 

40 

19 

Male 

340 

87 

174 

81 

Employee  Type 

Contractor 

99 

25 

97 

45 

Civilian 

3 

1 

1 

1 

Military 

289 

74 

116 

54 

Citizenship 

Born  U.S.  citizen 

377 

96 

190 

89 

Born  Abroad/US  Parents 

7 

2 

8 

4 

Naturalized 

7 

2 

16 

7 

4 


Data  Collection 


Efforts  were  made  to  ensure  that  data  gathered  for  this  study  were  of  high  quality. 
The  two  evaluators  who  reviewed  and  coded  SSBI-PRs  had  over  60  years  combined 
counterintelligence  and  investigative  expertise.  One  evaluator  worked  in  counterintelli¬ 
gence  for  the  Department  of  the  Army  Inspector  General  for  over  30  years  while  our 
other  evaluator  worked  in  counterintelligence  for  the  Air  Force  Office  of  Special  Investi¬ 
gations  for  over  30  years.  Both  individuals  had  7  years  of  experience  conducting  back¬ 
ground  investigations  at  the  time  of  our  study. 

Data  Collection  Software  Program 

Our  evaluators  gathered  data  from  case  files  using  a  Microsoft  Access  software 
program.  The  coding  program  was  designed  to  minimize  coder  fatigue  and  to  assure  data 
completeness  and  accuracy.  The  data  collection  program  notified  our  evaluators  if  re¬ 
quired  information  was  missing  or  if  entries  were  inconsistent  with  the  type  or  range  of 
expected  values.  Materials  that  were  used  to  train  coders  are  shown  in  Appendix  A. 

The  Coding  Process 

Phase  1  SSBI-PRs  are  those  that  DSS  said  did  not  contain  issue-relevant  informa¬ 
tion  in  Phase  1  sources.  Phase  2  SSBI-PRs  are  cases  that  DSS  investigators  felt  contained 
issue-relevant  information  in  Phase  1  sources  and  which  were  therefore  expanded  to 
include  Phase  2  sources. 

The  coding  process  entailed  documenting  the  following  for  Phase  1  SSBI-PRs: 

•  Did  Phase  1  sources  yield  issue-relevant  information  in  the  investiga¬ 
tion?  (In  other  words,  did  our  evaluators  agree  with  DSS’  Phase  1 
designation?) 

•  If  Phase  1  sources  yielded  issue-relevant  information,  which  sources 
yielded  this  information  and  what  was  the  nature  of  this  information? 

•  Did  Phase  2  sources  yield  issue-relevant  information?  (In  other  words, 
if  Phase  2  sources  had  been  used,  what  issue-relevant  information 
would  they  have  yielded?) 

•  If  Phase  2  sources  yielded  issue-relevant  information,  which  sources 
yielded  this  information  and  what  was  the  nature  of  this  information? 

The  coding  process  entailed  documenting  the  following  for  Phase  2  SSBI-PRs: 

•  Did  Phase  1  sources  yield  issue-relevant  information  in  the  investiga¬ 
tion?  (In  other  words,  did  our  evaluators  agree  with  DSS’  Phase  2 
designation?) 
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•  If  Phase  1  sources  did  not  yield  issue-relevant  information,  what 
derogatory  information  was  yielded  by  Phase  1  sources  such  that  DSS 
decided  to  expand  the  investigation? 

•  Did  Phase  2  sources  yield  issue-relevant  information  in  the  investiga¬ 
tion?  (While  this  information  would  not  have  been  missed,  as  Phase  2 
sources  are  used  in  Phase  2  investigations,  we  wanted  to  be  able  to 
estimate  the  productivity  of  Phase  2  sources  in  Phase  2  investigations.) 

•  If  Phase  2  sources  yielded  is  sue -relev ant  information,  what  sources 
yielded  this  information  and  what  was  the  nature  of  this  information? 

Sources  Used  in  SSBI-PRs  and  Coded  for  This  Study 

As  suggested  above,  sometimes  sources  were  examined  as  a  group  and  sometimes 
they  were  coded  individually.  In  answering  the  general  question  “Did  Phase  1  sources 
yield  issue-relevant  information  in  the  investigation?”  our  evaluators  examined  all  Phase 
1  sources  as  a  group.  But  in  answering  the  more  specific  question  “If  Phase  1  sources 
yielded  issue-relevant  information,  which  sources  yielded  this  information  and  what  was 
the  nature  of  this  information?”  our  evaluators  reviewed  and  coded  sources  individually. 
Sources  used  in  SSBI-PRs  and  coded  for  this  study  are  shown  in  Table  3. 

Inter-Coder  Reliability  Assessment 

To  assess  coding  consistency,  a  test  of  inter-coder  reliability  was  conducted.  Each 
evaluator  completed  the  same  10  SSBIs  randomly  selected  from  our  sample  of  cases. 
Comparisons  of  coders’  responses,  when  coding  the  same  10  files,  showed  that  they 
agreed  in  their  coding  decisions  a  relatively  high  percentage  of  the  time.  Their  average 
level  of  agreement  for  all  variables  combined  was  93%.  Given  that  coding  required  that 
individual  judgment  be  exercised  in  all  coding  decisions,  this  is  a  favorable  level  of  inter¬ 
coder  agreement  (Landis  &  Koch,  1977;  Shrout  &  Fleiss,  1979;  Tinsley  &  Weiss,  1975). 
Details  concerning  the  inter-coder  reliability  assessment,  including  our  evaluators’  level 
of  agreement  for  specific  types  of  variables,  are  shown  in  Appendix  B. 
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Table  3 

Sources  Used  in  SSBI-PRs  and  Coded  for  this  Study 


Phase  1  Sources 

•  Electronic  Personnel  Security  Questionnaire 

•  Subject  Interview 

•  Credit  Report 

•  Education  Records:  transcripts,  diplomas,  transcripts,  etc. 

•  Employment  Interviews:  interviews  with  Subject’s  current  coworker  or  supervisor 

•  Employment  Records:  performance  evaluations,  work  attendance  records,  letters 
of  reprimand,  etc. 

•  Ex-Spouse  Interview:  interview  of  former  spouse/cohabitant 

•  Local  Agency  Checks  (LAC):  checks  of  records  held  by  police  departments, 
sheriffs  departments,  county  courts,  etc. 

•  Medical  Interviews:  interviews  of  therapists,  social  workers,  psychiatrists,  etc. 

•  Medical  Records:  case  files,  treatment  records,  etc. 

•  NAC/Subject:  national  agency  checks  for  the  Subject 

•  NAC/Spouse-Cohabitant:  national  agency  checks  for  the  Subject’s  spouse  or 
cohabitant 

•  Other  Interviews:  confidential  sources,  security  manager  interviews,  interviews 
not  categorized  elsewhere 

•  Other  Records:  military  service  records,  public  records  (e.g.,  verifications  of 
divorce,  bankruptcy,  other  court  actions),  Title  3 1  records  (large  currency  transac¬ 
tion  reports),  security  records,  and  records  not  fitting  any  other  records  category 

Phase  2  Sources 

•  Developed  References:  interviews  of  references  who  are  developed  through  other 
interview  sources 

•  Listed  References:  interviews  of  references  listed  on  Subject's  EPSQ 

•  Residence  Interview:  interviews  of  Subject’s  past  or  present  neighbors 

•  Residence  Records:  lease  agreements,  rental  records,  etc. 

a 

Phase  1  and  Phase  2  sources  are  distinguished  based  upon  prior  research  in  which  this  phasing  model  was 
found  to  be  optimal.  For  more  information  about  this  phasing  model,  and  results  of  phasing  analyses 
conducted  with  SSBI-PRs,  see  SSBI-PR  Source  Yield:  An  Examination  of  Sources  Contacted  During  the  SSBI- 
PR  (Heuer,  Crawford,  Kramer  &  Hagen,  2001). 
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Results 


Percentage  of  SSBI-PRs  Expanded  by  DSS 

Using  the  expansion  criterion  of  issue-relevant  information  in  one  or  more 
Phase  1  sources  (the  expansion  criterion  used  in  earlier  phasing  studies),  2,341  (15%) 
of  the  15,668  SSBI-PRs  opened  and  closed  between  October  1,  2002  and  September  30, 
2003  were  expanded  by  DSS.  Conversely,  85%  of  these  SSBI-PRs  were  deemed  to 
contain  no  issue-relevant  information  in  Phase  1  sources  and  thus  were  not  expanded 
(Table  4). 


Table  4 

Percentage  of  SSBI-PRs  Expanded  by  DSS 
(N =15, 668) 


Type  of 

SSBI-PRs 

Investigation a 

n 

% 

Phase  1 

13,327 

85 

Phase  2 

2,341 

15 

Total 

15,668 

100 

a 

Phase  1  SSBI-PRs  contain  Phase  1  sources  only.  Phase  2  SSBI- 
PRs  contain  both  Phase  1  and  Phase  2  sources. 


Our  analyses  showed  that  our  evaluators  would  have  expanded  a  very  similar  per¬ 
centage  of  investigations,  despite  the  fact  that  they  would  have  expanded  some  SSBI-PRs 
that  DSS  did  not  expand,  and  would  not  have  expanded  some  SSBI-PRs  that  DSS  did  ex¬ 
pand  (as  analyses  presented  later  in  this  report  will  show).  Our  evaluators  would  have  ex¬ 
panded  16%  of  SSBI-PRs  in  comparison  with  DSS’  expansion  rate  of  15%. 

Percentage  of  Phase  1  SSBI-PRs  Where  Phase  2  Sources  Yielded  Issue-Relevant 
Information 

In  387  of  the  391  SSBI-PRs  that  DSS  designated  as  Phase  1  investigations  (but 
for  which  they  subsequently  conducted  Phase  2  sources),  Phase  2  sources  would  have 
yielded  no  issue -relevant  information  if  they  had  been  used  (Table  5).  Conversely,  there 
were  four  SSBI-PRs  that  DSS  designated  as  Phase  1  in  which  Phase  2  sources  would 
have  yielded  issue-relevant  information  if  they  had  been  used  (less  than  1%  of  Phase  1 
cases).  (DSS  conducted  Phase  2  leads  on  the  sample  of  Phase  1  cases  after  they  were 
closed  so  we  could  determine  if  Phase  2  sources  would  have  yielded  issue-relevant 
information.) 
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Table  5 

Percentage  of  Phase  1  SSBI-PRs  Where  Phase  2 
Sources  Yielded  Issue-Relevant  Information 
(n=391) 


Phase  2  Sources  Would  Have  Yielded 

Phase  1 

SSBI-PRs 

No  Issue-Relevant 

Issue-Relevant 

Information 

Information 

391 

387 

4 

100% 

99% 

<1% 

The  issue-relevant  information  that  would  have  been  yielded  by  Phase  2  sources 
in  these  four  SSBI-PRs  is  described  below.  This  information  was  considered  by  our 
evaluators  to  be  of  limited  security  significance. 

•  Case  1 

Developed  Reference:  Revealed  that  Subject  had  recently  married  a  citizen  of  the 
United  Kingdom. 

Listed  Reference:  Revealed  that  Subject  is  marrying  a  citizen  of  the  United 
Kingdom. 

Ex-Spouse  Interview:  Advised  that  Subject  employed  a  nanny  from 
Latvia  for  two  periods  while  he  was  divorced.  He  kept  contact  with  the  nanny  and 
had  visited  her  in  2003  in  Latvia.  Subject  told  ex-spouse  that  he  had  previously 
been  engaged  to  a  woman  from  Russia  but  that  the  relationship  ended  several 
years  ago. 

•  Case  2 

Residence  Interview:  Neighbor  mentioned  Subject  may  have  been  behind  on 
some  bills,  but  thought  it  could  be  due  to  a  mix-up  in  the  mail. 

•  Case  3 

Developed  Reference:  As  a  possible  indicator  of  undue  affluence,  a  developed 
reference  indicated  that  Subject  spends  a  lot  of  money. 

•  Case  4 

Developed  Reference:  Reported  hearing  that  Subject  had  participated  in  a 
program  related  to  alcohol  abuse  while  stationed  in  Japan. 

Percent  Evaluator  Agreement  with  DSS’  Phase  1  Designation 

Our  evaluators  felt  that  Phase  1  sources  did  not  yield  issue-relevant  information  in 
370  of  the  391  SSBI-PRs  that  DSS  investigators  designated  as  Phase  1,  but  would  have 
expanded  21  SSBI-PRs  that  DSS  did  not  expand.  Thus,  our  evaluators  agreed  with  DSS’ 
Phase  1  designation  95%  of  the  time  and  disagreed  with  DSS’  Phase  1  designation  5%  of 
the  time  (Table  6). 
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Table  6 

Percent  Evaluator  Agreement  with 
DSS  Phase  1  Designation 


Phase  1  SSBI-PRs 

Evaluator  Agreed  with 

Evaluator  Disagreed  with 

According  to  DSS 

Phase  1  Designation 

Phase  1  Designation 

391 

370 

21 

100% 

95% 

5% 

The  issue-relevant  information  yielded  by  Phase  1  sources  in  10  cases  of  the  21 
SSBI-PRs  that  our  evaluators  would  have  expanded  (but  which  DSS  did  not  expand),  is 
described  below.  Rather  than  describing  the  information  for  all  21  SSBI-PRs,  ten  exem¬ 
plary  cases  are  provided. 

•  Case  1 

Credit  Report:  Showed  two  bad  debts  and  three  unpaid  collections,  totaling 
$1,600  one  year  prior  to  investigation. 

Subject  Interview:  Subject  admitted  to  becoming  financially  overextended  in  use 
of  credit  cards. 

•  Case  2 

Ex-Spouse  Interview:  Ex- spouse  stated  he  had  "nothing  good"  to  say  about 
Subject,  and  it  would  "be  in  her  best  interest  if  he  were  not  interviewed.” 

•  Case  3 

EPSQ:  Subject’s  spouse  is  a  Malaysian  citizen  to  whom  he  has  been  married  since 
1990.  EPSQ  showed  travel  to  Malaysia  by  Subject  and  spouse  to  visit  family. 

•  Case  4 

Subject  Interview:  Subject  admitted  counseling  for  anxiety  attacks  and  that  he  is 
currently  taking  psychotropic  medications. 

•  Case  5 

Credit  Report:  Credit  report  in  connection  with  the  ESPQ  shows  falsification  of 
the  report.  Credit  report  also  shows  a  pattern  of  financial  irresponsibility. 

EPSQ:  Subject  denied  any  delinquencies  in  excess  of  180  days  but  credit  report 
shows  two  recent  collections/charge-offs  totaling  $1,060. 

•  Case  6 

EPSQ:  Subject’s  wife  is  a  German  citizen.  Subject  listed  annual  travel  to 
Germany.  His  four  children  are  listed  as  German  citizens. 

Subject  Interview:  Subject  reported  that  he  has  traveled  to  Germany  annually 
for  4  weeks  each  time  to  visit  his  wife's  relatives  in  Germany. 
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•  Case  7 

Subject  Interview:  Subject  admitted  mental  health  counseling  for  depression  and 
taking  Prozac  for  that  condition. 

•  Case  8 

Ex-Spouse  Interview:  Ex- spouse  alleged  11  years  of  marital  infidelity. 

•  Case  9 

Public  Records:  Divorce  records  contain  an  allegation  by  his  ex-spouse  that  the 
marriage  dissolved  because  of  excessive  alcohol  use  and  violence  by  Subject. 
Subject  Interview:  Subject  admitted  drinking  four  to  five  beers  once  per  week  but 
denied  having  alcohol  problems.  Subject  stated  his  ex-spouse  was  the  daughter  of 
an  alcoholic  and  considered  any  alcohol  use  excessive.  Subject  stated  the  only 
violence  in  their  marriage  was  due  to  his  attempts  to  discipline  his  son  who  was  a 
"problem  child"  and  that  the  son  also  employed  violence. 

•  Case  10 

Credit  Report:  Subject  has  two  credit  reports.  The  first  credit  report  showed 
personal  debt  of  about  $45,000  and  $130,000  mortgage.  The  second  credit  report 
showed  $85,000  in  debt  with  the  same  mortgage  and  several  bad  debts. 

In  expanding  an  additional  21  SSBI-PRs  that  DSS  did  not  expand,  our  evaluators 
would  have  lowered  DSS’  misses  by  three  cases.  Three  of  the  four  cases  that  DSS  desig¬ 
nated  as  Phase  1,  but  in  which  Phase  2  sources  would  have  been  productive,  would  have 
included  these  sources  if  our  evaluators  had  made  the  phasing  decision.  By  expanding  a 
slightly  greater  number  of  cases,  our  evaluators  would  have  reduced  the  number  of 
misses  substantially.  The  one  SSBI-PR  that  our  evaluators  would  not  have  expanded  in 
which  Phase  2  sources  would  have  yielded  issue-relevant  information  is  Case  1  immedi¬ 
ately  following  Table  5. 

Percent  Evaluator  Agreement  with  DSS’  Phase  2  Designation 

There  were  42  SSBI-PRs  that  DSS  designated  as  Phase  2  that  our  coders  did  not 
feel  contained  issue-relevant  information  in  Phase  1  sources.  Thus,  our  evaluators  agreed 
with  DSS’  Phase  2  designation  80%  of  the  time  and  disagreed  with  DSS’  Phase  2  desig¬ 
nation  20%  of  the  time  (Table  7). 
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Table  7 

Percent  Evaluator  Agreement  with 
DSS’  Phase  2  Designation 
(n=214) 


Phase  2  SSBI-PRs 

Evaluator  Agreed  with 

Evaluator  Disagreed  with 

According  to  DSS 

Phase  2  Designation 

Phase  2  Designation 

214 

172 

42 

100% 

80% 

20% 

The  derogatory  information  contained  in  Phase  1  sources  in  10  cases  of  the  42 
SSBI-PRs  that  DSS  expanded,  but  which  our  evaluators  would  not  have  expanded,  is  de¬ 
scribed  below.  DSS  felt  the  information  was  issue-relevant,  but  as  our  evaluators’  re¬ 
marks  suggest,  our  evaluators  did  not  concur.  Rather  than  describing  all  42  SSBI-PRs,  10 
examples  are  provided. 

•  Case  1 

Subject  Interview:  Subject's  mother  and  brother  in  Barbados  are  not  an  issue. 

•  Case  2 

Subject  Interview:  Subject's  spouse  is  naturalized.  Her  Korean  family  is  not  an 
issue. 

•  Case  3 

Subject  Interview:  Subject's  ties  to  his  brother  and  other  relatives  in  Hungary  are 
not  grounds  for  an  issue.  Subject  denies  ability  to  be  influenced,  and  has  no 
financial  or  other  bonds. 

•  Case  4 

EPSQ:  DUI  in  1984  was  previously  adjudicated.  No  further  information  in  current 
investigation.  Not  an  issue. 

•  Case  5 

LACs:  Assault  arrest  in  Arizona  is  clearly  not  the  Subject  in  question.  There  was 
no  match  on  name,  SSN,  date  or  place  of  birth,  or  location.  No  issue. 

Subject  Interview:  Subject  has  siblings  in  India  and  UK,  but  no  indications  of 
close  associations  or  contact.  No  issue. 

•  Case  6 

Subject  Interview:  Subject  received  counseling  for  depression  and  marital  issues 
for  a  few  sessions.  He  took  no  medication  and  there  is  no  continuing  counseling. 
Marital  situation  was  resolved.  No  issue  of  current  concern. 
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•  Case  7 

Subject  Interview:  Subject  reported  involvement  in  a  security  incident  as  a 
witness,  but  clearly  stated  he  was  not  held  responsible  or  subject  to  adverse 
action.  No  issue. 

•  Case  8 

Employment  Interviews:  Subject's  supervisor  reported  a  security  violation  which 
was  not  intentional  and  did  not  result  in  compromise. 

Subject  Interview:  Subject  has  one  security  violation  which  was  not  intentional 
and  did  not  result  in  compromise.  This  information  should  not  have  triggered 
Phase  2. 

•  Case  9 

Subject  Interview:  Forgot  to  list  one  military  exercise  related  travel  to  Korea 
during  initial  subject  interview.  This  is  completely  irrelevant. 

•  Case  10 

Subject  Interview:  Subject  received  stress  management  counseling  after  having  a 
heart  attack.  Subject  takes  no  medicine  and  demonstrates  no  failure  to  follow 
treatment.  No  issue. 

Of  the  42  SSBI-PRs  that  our  coders  would  not  have  expanded  to  include  Phase  2 
sources,  there  was  one  in  which  Phase  2  sources  did  yield  issue-relevant  information.  In 
this  case  a  listed  reference  indicated  that  the  Subject  received  mental  health  counseling 
for  depression  and  was  taking  Prozac.  The  personnel  security  questionnaire  (a  Phase  1 
source)  indicated  that  the  Subject  received  mental  health  counseling  but  only  for  marital 
purposes.  Mental  health  counseling  was  not  identified  as  a  security  issue  in  the  subject 
interview  either  (another  Phase  1  source).  It  is  not  possible  to  determine  whether  the 
listed  reference,  the  Subject,  or  both  the  reference  and  the  Subject  provided  accurate  in¬ 
formation.  This  case  illustrates  the  importance  that  Phase  1  sources — especially  the  sub¬ 
ject  interview — be  conducted  effectively  and  written  up  in  detail  when  warranted. 

Reduction  in  Resource  Requirements  Resulting  from  DSS’  Implementation  of  a 
Phased  SSBI-PR 

Using  data  gathered  for  a  productivity  study  conducted  by  DSS  (Mitchell,  1999), 
data  gathered  for  the  initial  phasing  study  (Heuer,  Crawford,  Kramer  &  Hagen,  2001), 
and  recent  analyses  conducted  to  estimate  SSBI-PR  time  requirements  (Crawford  & 
Kramer,  2003),  we  estimated  the  reduction  in  resource  requirements  that  resulted  from 
DSS’  implementation  of  a  two-phase  SSBI-PR  in  FY03. 

Analyses  based  on  112,928  SSBI-PRs  (Mitchell,  1999)  showed  listed  reference 
interviews,  developed  reference  interviews,  and  residence  interviews  (three  of  our  four 
Phase  2  sources)  to  constitute  approximately  42%  of  the  total  cost  of  conducting  an 
SSBI-PR.  (Cost  data  for  residence  records  checks  were  unavailable  and  are  therefore  not 
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considered  in  our  analysis.)  When  these  cost  data  were  gathered,  however,  SSBI-PR  ref¬ 
erence  coverage  requirements  differed  from  those  in  place  in  FY03  at  DSS. 

In  early  2002,  reference  requirements  for  SSBI-PRs  were  revised  such  that 
individuals  could  qualify  as  more  than  one  type  of  reference  if  the  Subject  of  the  investi¬ 
gation  knew  the  individual  in  more  than  one  capacity.  Analyses  conducted  for  our  initial 
phasing  study  suggest  that  this  change  resulted  in  a  reduction  of  approximately  1.4  Phase 
2  interviews  per  SSBI-PR.  Whereas  DoD  conducted  an  average  of  3.8  Phase  2  interviews 
in  2001  for  each  SSBI-PR,  under  a  policy  similar  to  that  later  enacted  for  DSS  investiga¬ 
tions,  OPM  conducted  approximately  2.4  Phase  2  interviews  per  SSBI-PR  (Heuer,  Craw¬ 
ford,  Kramer  &  Hagen,  2001,  p.  19).  This  reduction  in  the  number  of  Phase  2  sources 
used  in  SSBI-PRs  has  implications  for  the  cost  savings  that  resulted  from  the  implemen¬ 
tation  of  a  phased  SSBI-PR. 

To  update  our  cost  figure  of  42%  to  reflect  the  reduction  in  the  number  of  Phase  2 
interviews  conducted  on  average  (and  the  fact  that  the  proportion  of  cost  constituted  by 
Phase  2  sources  is  reduced  as  a  result),  the  original  figure  of  42%  was  multiplied  by  63% 
(2.4  interviews  divided  by  3.8  interviews  =  .63)  to  arrive  at  estimated  resource  savings  of 
26%  for  each  SSBI-PR  that  does  not  include  Phase  2  sources. 

Thus,  in  implementing  phasing  in  FY03,  DSS  reduced  resource  requirements  by 
26%  in  the  85%  of  periodic  reinvestigations  they  did  not  expand.  This  resulted  in  an 
overall  reduction  in  resource  requirements  of  conducting  SSBI-PRs  in  FY03  of  22%  (.85 
multiplied  by  .26  =  .22).  That  is  to  say,  whatever  the  cost  of  conducting  the  15,668  peri¬ 
odic  reinvestigations  in  FY03  would  have  been  without  the  implementation  of  phasing,  in 
implementing  a  phased  SSBI-PR,  DSS  reduced  these  resource  requirements  by  22%. 

Table  8 

Reduction  in  Resource  Requirements  Resulting  from 
DSS’  Implementation  of  a  Phased  SSBI-PR 


Proportion  of 

Reduction  in  Resources 

Total  Resource 

SSBI-PRs  Not 

for  Each  SSBI-PR 

Savings  for  the 

Expanded 

Not  Expanded 

SSBI-PRs  FY03 

85% 

26% 

22% 

Conclusions  and  Recommendations 

The  findings  of  this  evaluation  support  findings  of  previous  phasing  studies  in 
which  a  two-phase  SSBI-PR  is  shown  to  result  in  a  substantial  reduction  in  resource  re¬ 
quirements  with  minimal  loss  of  derogatory  information.  In  the  small  number  of  SSBI- 
PRs  that  were  not  expanded  by  either  DSS  investigators  or  by  our  evaluators  and  in 
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which  Phase  2  sources  would  have  been  productive  if  they  had  been  used,  the  informa¬ 
tion  that  was  missed  was  relatively  minor. 

Findings  of  the  current  study  support  previous  research  in  which  phasing  is  shown 
to  offer  tremendous  benefits  with  minimal  risk  even  when  there  is  some  level  of  dis¬ 
agreement  as  to  when  SSBI-PRs  should  be  expanded.  Phasing  has  operated  effectively 
when  DSS  investigators  have  made  the  phasing  decision  (as  in  the  current  study  and  in 
the  DoD  pilot  test  conducted  in  2001)  and  also  when  different  sets  of  evaluators  have 
made  the  phasing  decision. 
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Appendix  A 

Coder  Training  Materials 


A.  Coding  Program 

A  software  program  is  used  to  code  the  SSBI-PRs  in  this  study.  For  each  investigation,  one  data 
entry  screen  is  completed.  This  data  entry  screen  is  shown  below. 


B  Phasing  Evaluation  Study  -  [Data  Entry  :  Form] 


liEl  File  Edit  View  Insert  Fermat  Records  Iools  Window  tdelp 

*]i  Tahoma  -*8  -»|  ■  1  y  [g  g  M  1 •  I  I 

h  - 1  a  ^  I  a  a  v  x  Efe  e,  q  %.  ^  i  v  »  >*  *  es1  b  a  -  ® . _ 

Case  #  |  l  SSN:  I  -  -  l  Case  Type:  Phase  HT 


JsJxJ 

Type  a  question  for  help  J  -  s  x 


***This  Case  is  Not  Complete  *** 


Issue  Relevant  Info? 


Sources  and  Issue  Rationales 


Phase  1  Sources 

U Yes £n° 

EPSQ 

Subject  Interview 
Credit  Report 
NAC-Subject 
NAC-Spouse 
LACs 

Ex-spouse  Interview 
Military  Records 
Security  Records 
Title  31  Records 


Employment  Interviews 
Employment  Records 
Education  Records 
Public  Records 
Misc,  Records 
Misc.  Interviews 
Medical  Interviews 
Medical  Records 
Area  Security  Manager 


Phase  2  Sources 

yes  ®  No 

Listed  References 
Developed  References 
Residence  Interviews 
Residence  Records 


Source 


Description 


|epsq 


EPSQ  showed  Subject  received  counseling  for  a 
substance  use  problem, 


Source 


Description 
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No  Issue 

EPSQ 

J 

Subject  Interview 

Credit  Report 
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NAC-Subject 
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Ex-Spouse  Interview 

Military  Records 
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Employment  Interviews 

Employment  Records 

Education  Records 

Public  Records 

Miscellaneous  Records 

Miscellaneous  Interviews 

Medical  Interviews 

Medical  Records 

Area  Security  Manager 

Show  All  Records 


Show  Incomplete  Only 


m  I'm  done  w/case 
i®  This  is  a  bad  case 


Record  1  of  30 


Previous  Next 

Record  Record 


Exit 


Record:  jiljJt 
Form  View 
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Microsoft 


Elements  of  the  Data  Entry  Screen 

•  Case  #:  shows  the  case  number  (which  should  match  the  number  at  the  top  of  the 
investigative  file). 

•  SSN:  shows  the  Social  Security  Number,  which  can  be  used  as  a  cross-check  against 
the  SSN  on  the  investigative  file. 
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•  Case  Type:  Shows  either  “Phase  1”  or  “Phase  2”.  Case  type  affects  the  behavior  of  the 
issue  description  field  for  Phase  1  sources.  Phase  1  cases  will  require  that  information  be 
entered  in  the  issue  description  field  if  the  “Phase  1  Sources”  radio  box  is  checked  “Yes” 
and  Phase  2  cases  will  request  information  in  the  description  field  if  the  “Phase  1 
Sources”  radio  box  is  checked  “No”.  Note  that  the  label  over  the  field  will  change 
depending  on  whether  issue  descriptions  are  required  or  an  explanation  of  why  there  is 
no  issue-relevant  information  is  required. 

•  ***This  Case  is  Not  Complete***:  This  message  appears  when  the  conditions  for  a  case 
to  be  called  complete  have  not  been  met.  These  conditions  are  the  following:  both  radio 
boxes,  Phase  1  Sources  and  Phase  2  Sources,  have  been  checked  either  yes  or  no,  and 
any  visible  issue  description  fields  have  at  least  1  issue  description. 

•  Issue-Relevant  Info  in  Phase  1  Sources?  /  Issue-Relevant  Info  in  Phase  2  Sources? 

Radio  buttons  are  used  to  indicate  whether  Phase  1  sources  or  Phase  2  sources  yielded 
issue-relevant  information  in  the  case. 

•  Issue  Description  Fields:  These  fields  become  visible  depending  on  whether  the  case  is 
a  Phase  1  investigation  or  a  Phase  2  investigation  and  whether  you  clicked  “Yes”  or  “No” 
for  “Issue-Relevant  Info?”  When  the  text  boxes  are  visible,  the  program  expects  at  least 
one  record  to  be  completed  requiring  the  selection  of  a  source  type  and  a  text 
description. 

o  Deleting  a  source  and  text:  A  source  type  and  text  description  may  be  deleted  by 
clicking  on  the  rectangle  on  the  left  side  of  the  record.  (In  the  screen  shot  above,  the 
record  with  the  “Ex-spouse  Interview”  has  a  small  triangle  in  the  rectangle,  pointing  at 
the  “Source”  label.)  If  there  is  information  in  the  issue  description  boxes  and  a  radio 
button  setting  is  changed,  the  text  fields  will  disappear  and  the  program  will  ask  you 
to  delete  the  text  related  to  the  radio  button.  If  the  user  confirms  this  deletion,  the  de¬ 
scriptions  will  be  deleted  permanently. 

•  Show  All  Records:  This  button  makes  all  records  accessible  to  the  coder. 

•  Show  Incomplete  Only:  This  button  makes  accessible  only  those  records  for  which  the 
coder  has  not  indicated  that  he  or  she  is  done  with  the  case. 

•  I’m  done  w/case:  When  the  “***This  Case  is  Not  Complete***”  label  is  opt  visible  the 
coder  may  check  the  “I’m  done  w/case”  checkbox  to  indicate  that  coding  for  the  case  is 
complete.  Note  that  this  is  not  equivalent  to  the  “***This  Case  is  Not  Complete***”  label 
since  it  is  possible  that  the  conditions  for  having  a  complete  case  have  been  met,  but 
there  is  still  information  in  the  case  that  needs  to  be  recorded.  Beware  of  the  temptation 
to  believe  that  a  case  has  been  completely  coded  just  because  the  “***This  Case  is  Not 
Complete***”  label  has  disappeared. 

•  This  is  a  bad  case:  This  box  may  be  checked  at  any  time  to  indicate  that  the  case  con¬ 
tains  insufficient  information  to  code,  or  should  be  excluded  for  similar  reasons. 

•  Previous  Record  and  Next  Record:  these  buttons  have  the  same  functionality  as  the 
record  controls  in  the  lower  left  corner. 

•  Exit:  This  button  quits  the  program.  (Coder  does  not  need  to  save  the  data,  this  is  done 
automatically.) 
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B.  Coding  Process 


Phase  1  SSBI-PRs  are  those  that  DSS  said  did  not  contain  issue-relevant  information  in  Phase  1 
sources,  and  thus  would  not  have  been  expanded  to  include  Phase  2  sources.  (Because  we 
wanted  to  know  what  issue-relevant  information  Phase  2  sources  would  have  yielded  in  these 
cases,  DSS  conducted  these  leads  after  these  investigations  were  closed.  Even  though  Phase  2 
leads  wouldn't  be  used  in  Phase  1  cases  in  reality,  we  have  both  Phase  1  and  Phase  2  leads  for 
these  investigations.) 

Phase  2  SSBI-PRs  are  cases  that  DSS  said  *did*  contain  issue-relevant  information  in  Phase  1 
sources.  DSS  conducted  both  Phase  1  and  Phase  2  leads  on  these  cases  also. 

In  evaluating  DSS'  implementation  of  phasing,  this  is  our  coding  process: 

Phase  1  SSBI-PRs 


•  Did  Phase  1  sources  yield  issue-relevant  information  in  the  case? 

If  “No”  additional  coding  is  riot  required 

If  "Yes"  the  issue-relevant  information  is  documented  in  the  window  that  pops  up. 

•  Did  Phase  2  sources  yield  issue-relevant  information  in  the  case? 

If  “Yes”  the  information  is  documented. 

If  “No”  additional  coding  is  not  required. 

Phase  2  SSBI-PRs 


•  Did  Phase  1  sources  yield  issue-relevant  information  in  the  case? 

If  "No"  the  derogatory  information  is  documented  and  a  rationale  for  why  it  is  riot  issue¬ 
relevant  information  is  provided. 

If  "Yes"  additional  coding  is  not  required. 

•  Did  Phase  2  sources  yield  issue-relevant  information  in  the  case? 

If  "Yes"  the  information  is  documented. 

If  "No"  additional  coding  is  not  required. 

As  indicated,  the  coding  process  for  Phase  2  sources  in  either  type  of  investigation  is  the  same:  if 
information  is  there,  it  is  documented;  if  no  information  exists,  it  is  not  documented.  The  differ¬ 
ence  is  in  the  coding  of  Phase  1  sources.  Wherever  you  disagree  with  DSS,  the  disagreement  is 
documented. 

C.  Sources  Containing  Issue-Relevant  Information 

Sources  are  coded  at  the  case  level.  If  one  or  more  sources  of  a  type  yields  issue-relevant  infor¬ 
mation  in  the  investigation  and  the  information  is  to  be  documented,  the  source  is  selected  from 
the  drop-down  menu  only  once.  Source  categories  that  contain  more  than  one  lead  type  are 
shown  below. 

•  Local  Agency  Checks/Law  Enforcement  Checks  include  Police  Department  Records 
and  records  pertaining  to  criminal  behavior. 

•  Other  Interviews  include  Security  Manager  Interviews,  Self-Employment  Interviews, 
Unemployment  Interviews,  and  miscellaneous  interviews  not  categorized  elsewhere. 
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•  Other  Records  include  Security  Records,  Self  Employment  Records,  Probation  /  Parole 
Records,  Public  Records,  Unemployment  Records,  Court  Records  pertaining  to  civil 
processes,  and  miscellaneous  records  not  categorized  elsewhere. 


Issue-relevant  information  is  information  that  is  relevant  to  establishing  that  an 
issue  is  of  potential  current  security  concern. 


D.  Criteria  Used  to  Expand  PRs  by  DSS 

Examples  of  issue-relevant  information  are  provided  below.  There  are  many  kinds  of 
derogatory  information  that  will  meet  the  threshold  for  issue-relevant  information  that  are 
not  listed. 

1.  Allegiance 

An  individual  must  be  of  unquestioned  allegiance  to  the  United  States.  Any  information 
that  suggests  otherwise  is  issue-relevant  information.  Examples  of  issue-relevant  infor¬ 
mation  pertaining  to  Allegiance  can  include  the  following: 

•  Previous  or  current  involvement  (by  admission  or  allegation)  in  any  act  of  sabotage, 
espionage,  treason,  terrorism,  sedition,  or  any  other  act  whose  aim  it  is  to  overthrow 
of  alter  the  U.S.  government  or  any  of  its  states/subdivisions  by  unconstitutional 
means  to  include  force  or  violence. 

•  Associating  with  or  sympathizing  with  other  persons  or  organizations  that  advocate  or 
practice  the  above  conduct. 

•  Association  or  sympathy  with  persons  or  organizations  that  advocate  the  overthrow 
of  the  U.S.  Government 

•  Involvement  in  activities  which  unlawfully  advocate  or  practice  the  commission  of  acts 
of  force  or  violence  to  prevent  others  from  exercising  their  rights  under  the  Constitu¬ 
tion  of  laws  of  the  U.S.  or  of  any  state. 

2.  Foreign  Influence 

A  security  risk  may  exist  when  an  individual’s  immediate  family,  including  cohabitants, 
and  other  persons  to  whom  he  or  she  may  be  bound  by  affection,  influence,  or  obligation 
(e.g.,  mothers,  fathers,  brothers,  sisters,  aunts,  uncles)  are  not  citizens  of  the  United 
States  or  may  be  subject  to  duress.  Any  information  that  suggests  that  Foreign  Influence 
is  a  potential  security  concern  is  issue-relevant  information.  Examples  of  issue-relevant 
information  pertaining  to  Foreign  Influence  can  include  the  following: 

•  An  immediate  family  member,  or  a  person  to  whom  the  individual  has  close  ties  of 
affection  or  obligation,  is  a  citizen  of,  or  resident  or  present  in,  a  foreign  country 

•  Subject’s  foreign  background  potentially  causes  him  or  her  to  have  divided  loyalty 
between  his/her  U.S.  citizenship  and  loyalty 
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•  As  a  general  rule,  close  foreign  relatives,  friends,  business  interests,  or  travel  by 
itself,  is  not  issue-relevant  information.  Issue-relevant  information  is  information  that 
pertains  to  Subject’s  behavior,  loyalty,  or  vulnerability  to  foreign  influence. 

3.  Foreign  Preference 

When  an  individual  acts  in  such  a  way  as  to  indicate  a  preference  for  a  foreign  country 
over  the  United  States,  then  he  or  she  may  be  prone  to  provide  information  or  make 
decisions  that  are  harmful  to  the  interests  of  the  United  States.  Examples  of  issue¬ 
relevant  information  pertaining  to  Foreign  Preference  can  include  the  following: 

•  The  exercise  of  dual  citizenship  while  a  U.S.  citizen  (even  if  outside  of  period  of 
investigation). 

•  Possession  and/or  use  of  a  foreign  passport  while  a  U.S.  citizen  (even  if  outside  of 
period  of  investigation). 

•  Military  service  or  a  willingness  to  bear  arms  for  a  foreign  country  while  U.S.  citizen 
(even  if  outside  period  of  investigation). 

•  Accepting  educational,  medical,  or  other  benefits  from  a  foreign  country  while  a  U.S. 
citizen  (even  if  outside  period  of  investigation). 

•  Residence  in  a  foreign  country  to  meet  citizenship  requirements  while  a  U.S.  citizen 
(even  if  outside  period  of  investigation). 

•  Using  foreign  citizenship  to  protect  financial  or  business  interests  in  another  country 
while  a  U.S.  citizen  (even  if  outside  period  of  investigation). 

•  Seeking  or  holding  political  office  in  a  foreign  country  while  U.S.  citizen  (even  if 
outside  period  of  investigation). 

•  Voting  in  foreign  elections  while  U.S.  citizen  (even  if  outside  period  of  investigation). 

•  Performing  or  attempting  to  perform  duties,  or  otherwise  acting,  so  as  to  serve  the 
interests  of  another  government  in  preference  to  the  interests  of  the  U.S.  while  U.S. 
citizen  (even  if  outside  period  of  investigation). 

4.  Sexual  Behavior 

Sexual  behavior  is  a  security  concern  if  it  involves  a  criminal  offense,  indicates  a 
personality  or  emotional  disorder,  subjects  the  individual  to  coercion,  exploitation,  or 
duress,  or  reflects  lack  of  judgment  or  discretion.  Information  pertaining  to  a  Subject’s 
sexual  orientation  or  preference  does  not  by  itself  constitute  issue-relevant  information. 
Examples  of  issue-relevant  information  pertaining  to  Sexual  Behavior  can  include  the 
following: 

•  Sexual  behavior  of  a  criminal  nature  whether  or  not  Subject  has  been  prosecuted 

•  Compulsive  or  addictive  sexual  behavior  when  the  person  is  unable  to  stop  a  pattern 
of  self-destructive  or  high-risk  behavior  or  behavior  which  is  symptomatic  of  a  per¬ 
sonality  disorder. 

•  Sexual  behavior  that  causes  an  individual  to  be  currently  vulnerable  to  coercion, 
exploitation,  or  duress. 
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Sexual  behavior  of  a  public  nature  that  reflects  lack  of  discretion  or  judgment. 


5.  Personal  Conduct 

Conduct  involving  questionable  judgment,  untrustworthiness,  unreliability,  lack  of  candor, 
dishonesty,  or  unwillingness  to  comply  with  rules  and  regulations  could  indicate  that  the 
person  may  not  properly  safeguard  classified  information.  Examples  of  issue-relevant 
information  pertaining  to  Personal  Conduct  include,  but  are  not  limited  to,  the  following: 

•  Refusal  to  cooperate  with  required  security  processing  or  to  complete  required 
security  forms. 

•  Any  falsification,  dishonesty,  and/or  omission  issue. 

•  Reliable  unfavorable  information  provided  by  references  that  may  be  significant  to 
assessing  Subject’s  reliability  or  trustworthiness. 

•  Understatement  of  derogatory  information. 

•  Behavior  applies  to  all  areas  of  Subject’s  life,  to  include  the  workplace. 

•  General  behaviors:  Lack  of  willingness  to  comply  with  rules,  regulations  and  proce¬ 
dures;  dysfunctional  behavior  such  as  problems  controlling  anger;  malicious  actions 
to  get  even;  harassment  or  stalking;  intimidation;  shoplifting;  behavior  that  resulted  in 
the  issuance  of  restraining  order 

•  Work  Behaviors:  lying  to  cover  up  mistakes,  petty  theft,  padding  travel  vouchers  or 
expense  accounts,  serious  misuse  of  government  or  company  resources,  and 
behaviors  that  in  the  military  result  in  non-judicial  punishment  such  as  AWOL  or 
conduct  unbecoming  an  officer. 

6.  Financial  Considerations 

An  individual  who  is  financially  overextended  is  at  risk  of  having  to  engage  in  illegal  acts 
to  generate  funds.  Unexplained  affluence  is  often  linked  to  proceeds  from  financially  prof¬ 
itable  criminal  acts.  Examples  of  issue-relevant  information  pertaining  to  Financial  Con¬ 
siderations  can  include  the  following: 

•  Debts  currently  90  days  or  more  overdue  with  a  combined  past  due  amount  of  $2000 
or  more. 

•  Paid  previously  overdue  accounts  totaling  $2000  or  more  within  the  last  two  years. 

•  Bankruptcy  action  within  the  period  of  the  investigations. 

•  Evidence  of  long-term,  ongoing  financial  problems — even  those  that  may  exist  from 
outside  the  current  investigation  period. 

•  Deceptive  or  illegal  financial  practices. 

•  Unexplained  affluence  such  as  paying  off  large  debts,  ownership  of  cars  or  boats  out¬ 
side  of  Subject’s  known  ability  to  acquire,  purchasing  houses  or  property  with  no 
mortgage  or  loans,  losing  large  sums  by  gambling  with  no  apparent  negative  results, 
etc. 
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7.  Alcohol  Consumption 


Excessive  alcohol  consumption  often  leads  to  the  exercise  of  questionable  judgment, 
unreliability,  failure  to  control  impulses,  and  increases  the  risk  of  unauthorized  disclosure 
of  classified  information  due  to  carelessness.  Examples  of  issue-relevant  information 
pertaining  to  Alcohol  Consumption  can  include  the  following: 

•  Any  (even  one)  alcohol-related  arrest,  charge,  or  conviction. 

•  Any  (even  one)  admitted,  developed,  or  alleged  incident  of  illegal  alcohol-related 
behavior  that  did  not  result  in  arrest. 

•  Any  (even  one)  admitted,  developed,  or  alleged  use  of  alcohol  at  work  (except  lawful 
use  specifically  permitted  by  employer  that  did  not  result  in  binge  drinking,  fighting, 
impaired  driving,  loss  of  work,  or  other  problems. 

•  Any  (even  one)  admitted,  developed,  or  alleged  alcohol-related  problem  behavior 
dealing  with  family,  friends,  or  neighbors  (to  include  binge  drinking,  fighting,  impaired 
driving,  loss  of  work,  or  other  problems. 

•  Admitted,  developed,  or  alleged  professional  diagnosis  and/or  treatment  for  alcohol 
abuse  or  dependence. 

•  Admitted,  developed,  or  alleged  serious  alcohol  problem  that  has  not  been 
professionally  treated  (even  if  no  alcohol  problem  currently  exists). 

•  Any  consumption  of  alcohol  (admitted,  developed  or  alleged)  after  professional 
diagnosis  of  alcohol  dependence,  and  completion  of  rehabilitation  problem,  even  if 
the  last  treatment  occurred  outside  the  POI. 

•  Current  attendance  at  Alcoholics  Anonymous  meetings  is  favorable  information.  It  is 
not  a  basis  for  conducting  Phase  2  if  it  is  the  result  of  a  previous  alcohol  problem  that 
has  already  been  adjudicated  when  there  is  no  indication  of  an  alcohol  problem 
within  the  scope  of  the  SSBI.  It  is  important  to  establish  if  the  Subject  ever  continued 
to  drink  after  a  professional  diagnosis  of  alcoholism  or  alcohol  dependency. 

8.  Drug  Involvement 

Improper  or  illegal  involvement  with  drugs  raises  questions  regarding  an  individual's  will¬ 
ingness  or  ability  to  protect  classified  information.  Drug  abuse  or  dependence  may  impair 
social  or  occupational  functioning,  increasing  the  risk  of  an  unauthorized  disclosure  of 
classified  information.  Drugs  are  defined  as  mood  and  behavior  altering  substances  and 
include  drugs,  materials,  and  other  chemical  compounds  identified  and  listed  in  the  Con¬ 
trolled  Substances  Act  of  1970  (e.g.,  marijuana  or  cannabis,  depressants,  narcotics, 
stimulants,  and  hallucinogens),  and  inhalants  and  other  similar  substances.  Drug  abuse 
is  the  illegal  use  of  a  drug  or  use  of  a  legal  drug  in  a  manner  that  deviates  from  approved 
medical  direction.  Examples  of  issue-relevant  information  pertaining  to  Drug  Involvement 
can  include  the  following: 

•  Frequent  or  recent  use  any  narcotic,  depressant,  stimulant,  hallucinogen,  inhalant, 
cannabis,  etc. 

•  Frequent  or  recent  misuse  of  over-the-counter  or  prescription  drugs. 

•  Illegal  use  or  misuse  of  substances  (narcotics,  depressants,  stimulants,  hallucino¬ 
gens,  inhalants,  cannabis,  etc.)  while  employed  as  a  law  enforcement  officer,  prose- 
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cutor,  or  courtroom  official;  while  possessing  a  security  clearance;  or  while  in  a 
position  directly  and  immediately  affecting  the  public  safety. 


•  Involvement  in  the  illegal  purchase,  manufacture,  trafficking,  production,  transfer, 
shipping,  receiving,  or  sale  of  any  narcotic,  depressant,  stimulant,  hallucinogen,  or 
cannabis  for  own  intended  profit  or  that  of  another. 

•  Drug  related  felony  arrest  if  sentenced  to  incarceration  of  more  than  one  year, 
regardless  of  whether  time  was  actually  served. 

•  Professional  diagnosis/evaluation  of  drug  abuse/dependency;  OR  any  involvement  in 
a  drug  treatment  program. 

9.  Emotional,  Mental,  and  Personality  Disorders 

Emotional,  mental,  and  personality  disorders  can  cause  a  significant  deficit  in  an  individ¬ 
ual's  psychological,  social,  and  occupational  functioning.  These  disorders  are  of  security 
concern  because  they  may  indicate  a  defect  in  judgment,  reliability,  or  stability.  A  creden¬ 
tial  mental  health  professional  (e.g.,  clinical  psychologist  or  psychiatrist),  employed  by, 
acceptable  to,  or  approved  by  the  government,  should  be  utilized  in  evaluating  potentially 
disqualifying  and  mitigating  information  fully  and  properly,  and  particularly  for  consultation 
with  the  individual's  mental  health  care  provider.  Examples  of  issue-relevant  information 
pertaining  to  Emotional,  Mental,  and  Personality  Disorders  can  include  the  following: 

•  Any  single  report,  whether  or  not  confirmed  and  regardless  of  what  the  medical 
diagnosis  may  be,  that  Subject  has  demonstrated  behavior  that  is  or  can  be  inter¬ 
preted  as  emotionally  unstable,  dysfunctional,  antisocial,  high  risk,  violent,  compul¬ 
sive,  paranoid,  bizarre  or  other  unusual  or  inappropriate  behavior,  that  may  indicate  a 
psychological  problem. 

•  Any  single  report,  whether  or  not  confirmed,  that  Subject  has  failed  to  follow  treat¬ 
ment  advice  related  to  a  diagnosed  emotional,  mental,  or  personality  condition,  e.g., 
failure  to  take  prescribed  medication. 

•  Any  report  that  Subject  is  taking  or  has  taken  psychotropic  medication  for  a  mental 
health-related  condition. 

10.  Criminal  Conduct 

A  history  or  pattern  of  criminal  activity  creates  doubt  about  a  person's  judgment,  reliabil¬ 
ity,  and  trustworthiness.  Examples  of  issue-relevant  information  pertaining  to  Criminal 
Conduct  can  include  the  following: 

•  Any  information,  whether  confirmed,  that  Subject  committed,  was  arrested  for, 

was  charged  for,  and/or  was  convicted  of  any  (even  one)  criminal  offense  during  the 
period  of  investigation,  excluding  routine  traffic  violations.  This  includes  pending 
criminal  charges  and  dropped  charges.  (See  condition  #2,  below,  for  UCMJ 
offenses.) 

•  Courts  martial  or  other  disciplinary  proceedings  under  the  UCMJ,  to  include  non¬ 
judicial,  Captain’s  mast,  etc. 

•  Discharge  or  dismissal  from  the  Armed  Forces  under  dishonorable,  or  less  than 
honorable,  conditions  at  any  time  in  Subject’s  life. 
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Criminal  incidents,  arrests,  charges,  and/or  convictions  outside  the  period  of 
investigation. 


•  A  person  who  has  been  convicted  in  a  federal  or  state  court,  including  courts  martial, 
and  sentenced  to  imprisonment  for  a  term  exceeding  one  year,  or  who  was  dishon¬ 
orably  discharged  or  dismissed  from  the  U.S.  armed  forces,  may  not  be  granted  or 
have  renewed  access  to  classified  information. 

11.  Security  Violations 

Noncompliance  with  security  regulations  raises  doubt  about  an  individual's  trustworthi¬ 
ness,  willingness,  and  ability  to  safeguard  classified  information.  Examples  of  issue¬ 
relevant  information  pertaining  to  Security  can  include  the  following: 

•  One  or  more  security  violations  resulting  in  unauthorized  disclosure  (alleged  or  admit¬ 
ted)  of  classified  or  other  protected  information,  if  not  previously  adjudicated. 

•  One  or  more  deliberate  security  violation(s),  even  if  outside  the  period  of 
investigation. 

•  Single  or  multiple  security  violation(s)  due  to  negligence,  a  pattern  of  carelessness, 
or  inattention  to  security  procedures  that  is  not  deliberate  and  resulted  in  no  unau¬ 
thorized  disclosure. 

•  One  or  more  security  violations  resulting  from  Subject  being  under  the  influence  of 
alcohol  and/or  drugs  (prescribed  or  illegal). 

•  Any  report,  whether  or  not  confirmed,  or  any  Cl  indicator  related  to  the  secure 
handling  of  information,  including: 

o  Seeking  information  without  need  to  know. 

o  Deliberately  revealing  classified  or  other  protected  information  to  an  unauthorized 
person. 

o  Unauthorized  or  suspicious  copying  of  classified  or  other  protected  materials,  or 
keeping  such  materials  at  home  or  any  other  unauthorized  location. 

o  Lying  about  the  destruction  or  other  disposition  of  classified  or  other  protected 
materials. 

12.  Outside  Activities 

Involvement  in  certain  types  of  outside  employment  or  activities  is  of  security  concern  if  it 
poses  a  conflict  with  an  individual's  security  responsibilities  and  could  create  an 
increased  risk  of  unauthorized  disclosure  of  classified  information.  Examples  of  issue¬ 
relevant  information  pertaining  to  Outside  Activities  can  include  the  following: 

•  Any  service,  whether  compensated,  volunteer  or  employment  with  a  foreign  country. 

•  Any  service,  whether  compensated,  volunteer  or  employment  with  any  foreign 
national. 

•  Any  service,  whether  compensated,  volunteer  or  employment  with  a  representative  of 
any  foreign  interest. 
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•  Any  involvement  with  a  foreign,  domestic,  or  international  organization  or  person 
engaged  in  analysis,  discussion,  or  publication  of  material  on  intelligence,  defense, 
foreign  affairs,  or  protected  technology. 

13.  Misuse  Information  Technology  Systems 

Noncompliance  with  rules,  procedures,  guidelines,  or  regulations  pertaining  to  informa¬ 
tion  technology  systems  may  raise  security  concerns  about  an  individual's  trustworthi¬ 
ness,  willingness,  and  ability  to  properly  protect  classified  systems,  networks,  and  infor¬ 
mation.  Information  Technology  Systems  include  all  related  equipment  used  for  the 
communication,  transmission,  processing,  manipulation,  and  storage  of  classified  or  sen¬ 
sitive  information.  Examples  of  issue-relevant  information  pertaining  to  Misuse  of  IT 
Systems  can  include  the  following: 

•  Removal  or  duplication  of  any  hardware,  software,  or  media  from  an  IT  system  with¬ 
out  authorization  (when  specifically  prohibited  by  rules,  procedures,  guidelines,  or 
regulations). 

•  Introduction  of  hardware,  software,  or  media  into  any  IT  system  without  authorization, 
when  specifically  prohibited  by  rules,  procedures,  guidelines,  or  regulations. 

•  Negligent  or  lax  computer  security  habits  that  persist  despite  counseling  by  manage¬ 
ment. 

•  Repeated  failure  by  a  system  administrator  to  implement  or  maintain  standard  IT 
security  procedures  that  are  required  and  available. 

•  Any  information  security  violation,  whether  deliberate  or  negligent,  that  results  in 
serious  damage  to  the  national  security. 

E.  Sworn  Statements  are  not  available. 


In  all  cases  that  reference  a  sworn  statement,  the  Subject  Interview  is  coded  as  issue-relevant. 

F.  Inter-Rater  Reliability 

Discussion  of  cases  is  encouraged,  with  the  exception  of  the  inter-rater  reliability  cases.  Toward 
the  beginning  of  the  second  week  of  coding,  each  of  you  should  complete  (or  have  completed) 
the  following  cases:  72,  1 02,  1 54,  268,  329,  541 , 567,  626,  655,  and  702.  Because  these  cases 
were  randomly  selected  from  all  cases,  they  are  scattered  throughout  the  files.  Ed  will  automati¬ 
cally  code  cases  1 54,  268,  626  and  655  and  Dan  will  automatically  code  cases  72,1 02,  541  and 
567  simply  by  coding  the  cases  that  were  assigned.  Cases  329  and  702  haven't  been  assigned  to 
any  coder  yet. 
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Appendix  B 

Inter- Coder  Reliability  Assessment 


To  assess  coding  consistency,  a  test  of  inter-coder  reliability  was  conducted.  Ten 
cases  were  randomly  selected  and  were  coded  by  each  evaluator.  Replicating  the  coding 
process  for  data  collection,  coders  indicated  whether  a  given  case  was  “bad”  or  usable 
(incomplete  cases  were  not  coded),  and  if  the  case  was  usable,  indicated  whether  issue¬ 
relevant  information  was  present  in  Phase  1  and  Phase  2  leads.  When  issue-relevant 
information  was  yielded  by  sources,  depending  upon  the  type  of  case  and  whether  the 
sources  were  Phase  1  or  Phase  2  sources,  the  evaluators  indicated  which  sources  yielded 
the  information.  Coders  also  documented  the  type  of  information  that  sources  yielded  by 
entering  data  into  a  text  box,  but  their  write-ups  were  not  compared  or  included  in  our 
analysis  of  coder  agreement. 

For  example,  if  Coder  A  coded  a  case  as  “bad”  and  Coder  B  also  coded  this  case 
as  “bad,”  this  counted  as  agreement.  If  Coder  A  indicated  that  a  Phase  1  source  yielded 
issue-relevant  information  in  a  case,  and  Coder  B  also  indicated  that  a  Phase  1  source 
yielded  issue-relevant  information  in  the  case,  this  counted  as  agreement.  If  Coder  A 
indicated  that  Phase  2  sources  yielded  issue-relevant  information,  and  Coder  B  indicated 
that  Phase  2  sources  yielded  no  issue-relevant  information,  this  counted  as  disagreement. 
Each  of  these  variables  was  measured  separately.  For  example,  if  Coder  A  called  a  case  a 
“bad”  case,  and  Coder  B  coded  the  case  all  the  way  through,  our  two  coders  would  dis¬ 
agree  on  a  number  of  variables  for  that  file. 

The  extent  to  which  coders  agreed  in  their  coding  of  sources  for  the  10  test  files  is 
shown  in  Table  B.l.  For  example,  of  the  10  cases,  coders  agreed  in  their  responses  con¬ 
cerning  Phase  1  sources  80%  of  the  time.  For  Phase  1  and  Phase  2  sources  the  coders’ 
average  agreement  was  90%.  Given  that  coding  required  that  individual  judgment  be 
exercised  in  all  coding  decisions,  and  that  phasing  performs  well  even  when  there  is  some 
disagreement  among  phasing  decisionmakers,  this  is  an  acceptable  level  of  inter-coder 
reliability  (Fandis  &  Koch,  1977;  Shrout  &  Fleiss,  1979;  Tinsley  &  Weiss,  1975). 


Table  B.l 

Inter- Coder  Reliability  Assessment 


Source 

Percent  Agreement 

Phase  1  Sources  Yielded 
Issue-Relevant  Information 

80% 

Phase  2  Sources  Yielded 
Issue-Relevant  Information 

100% 

Bad  Case 

100% 
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